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ABSTRACT 

The research identified and explored the shared knowledge among the instructional multimedia design and 
development experts comprising of subject matter expert, graphic designer and instructional designer. The 
knowledge shared by the team was categorized into three groups of multimedia design principles encompasses 
of basic principles, authoring principles and design principles. The research focused on soliciting knowledge of 
agreement on the principles. The research design sequentially began with the modified Delphi technique which 
involved twelve experts in selecting the list of principles in multimedia design. Next phase, the interview 
session involved three selected experts to verify the list of principles and obtain detailed information. Outcome 
of the research is essential in providing description of the cognitive skills needed to perform tasks in multimedia 
design and development proficiently. 

1.0 INTRODUCTION 

A team Cognitive Task Analysis is helpful as it can describe the way the team is thinking as opposed to the steps 
it is following. Cognitive processes for teams consist of control of attention, shared situation awareness, shared 
mental models, application of strategies and heuristics to make decisions, solve problems and plan and 
metacognition (Cook et. al., 2001). Shared mental models imply that team members have the same 
understanding for the roles and functions of each team members in accomplishing the task, the nature of the 
task, the use of equipment and so forth. In most settings a critical factor is the degree to which the team 
members have a shared mental model of their own roles and functions (Schraagen, Chipman & Shalin, 2000). 

One common source of difficulty for teams is when the members are confused about who is supposed to do 
what. Confusion about roles and functions leads to wasted effort or a failure to carry out essential subtasks. 
Effective teams understand the functions, including the common routines. Hence shared mental models refer to 
the configuration of the team and the way it is supposed to perform routines (Hoffman & Militello, 2008). 

Another issue that must be addressed is how the elicited information is represented which have not received 
much attention in the literature, but are crucial if a true picture of team-level knowledge stemming from a team 
CTA are to be useful. This includes an understanding of what each team members needs to know to function 
effectively, as well as an understanding of what information must be dynamically shared among members 
(Schraagen, Chipman & Shalin, 2000) 

Research aims at addressing this issue is clearly needed. Despite the gaps in research, a number of knowledge 
elicitation methods are available from research since such data are required so that team selection, training, task 
design and management systems can be optimized. Thus, the research is focusing on soliciting knowledge of 
agreement on strategies (Schraagen, Chipman & Shalin., 2000). The research questions thus highlight the 
following issues: 

i) What are the tasks shared among graphic designers, instructional designers and subject matter experts 

in multimedia design and development for instructional purposes? 

ii) What are the tasks which are based according to their expertise in design and development for 

instructional purposes? 

2.0 REVIEW OF RELATED LITERATURE 
2.1 Multimedia Design and Development Experts 

Bergman and Moore (1991) describe development experts as comprising several specialists who perform the 
design, development, production and authoring work. They are divided into primary roles and supporting roles. 
The former consist of Application Designer, Managing Producer, Art Director and Video Director. Meanwhile 
the latter consist of Writers, Graphic Artists, Developers, Audio and Video Production Personnel, Authors, 
Programmers, Subject Matter Experts and Administrators. Bergman and Moore (1991) add that some 
individuals may have multiple skills, thus the team does not necessarily comprise of every specialization. 
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Careful selection of members with the right skills is essential as it is not realistic to expect any one member to 
possess all the sophisticated skills required. 

Lee and Owens (2000) argue that organizations that assume a couple of people with the right skill sets can 
design and produce multimedia are misinformed about the number and complexity of skill sets needed for even 
a minimal multimedia design and development project. Assigning roles to the project group depends on the size 
of the project, timeline, skills and resources needed. They describe the roles into eighteen, which is more than 
advocated by Bergman and Moore (1991), encompassing Audio Producer or Technician, Author (Publisher, 
Materials Developer), Creative Director, Editor, Evaluation Specialist, Graphic Artist, Graphic Designer, 
Implementation Representative, Instructional Designer (Interactive Designer), Performance Analyst, Project 
Manager (Project Leader), Quality Review (Evaluator), Sponsor, Subject-Matter Expert (SME), Systems 
Designer, Systems Engineer (Application Developer), Video Editor (Technician) and Video Producer. 

Despite emphasizing on the number of roles, Allessi and Trollip (2001) highlight the importance of 
brainstorming and collaboration among the experts as an excellent way of bringing together the different people 
and skills that can contribute productively to project design. Compared to the earlier researchers, Alessi and 
Trollip (2001) divide the team into a group that creates a design of the program for the most effective learning 
by the target audience and another group that produces or oversees the production of a set of documents for 
effective communication with the rest of the team. The former include the Instructional Designer, Clients, 
Project Manager, Content Experts, Trainers or Teachers and learners. Alessi and Trollip (2001) describe the 
latter as the production staff comprising technical writers, programmers, graphic artists, photographers, 
videographers, audiographers, special effects artists and actors. 

On the other hand, another group of researchers in multimedia have their own perspective with regard to the role 
of the multimedia development team. Jamalludin et al. (2003) categorized the team according to three major 
groups, namely: management, design and technical. The management group comprises of project manager, 
creative director, art director, technical director and test director. The design group includes subject matter 
expert, instructional designer and script writer. The technical group encompasses the editor, graphic computer 
artist, audio specialist, video specialist, 2D animator, 3D animator and programmer. Collaboration between the 
three groups determines the flow of the project and thus ensures multimedia product quality. 

Norazlin et al. (2007) agree to the notion of grouping the team into three different groups according to their area 
specialization. They add that the role depends on the needs of the particular project, since such needs differs 
between one project and another. Thus, some of the roles may not be applicable to another multimedia project. 
Apart from defining the roles, they also emphasize the working culture and leadership quality in the multimedia 
development team in determining the flow of the project for producing quality multimedia products. 

For the purpose of the research, the team of multimedia development is categorized generally into three main 
groups which are instructional designer, graphic designer and subject matter experts. Thus, this research 
focuses on the shared knowledge among the three groups in the team. 

3.0 RESEARCH DESIGN 

A cognitive task analysis was conducted using twelve multimedia design and development experts to construct 
the list of principles in designing multimedia-based instructional media. For the purpose of establishing content 
validity of the multimedia design principles, the number of experts is more than 10 due to its consistency with 
Dalkey’s finding (as cited in Martino, 1972). The sample involved experts from various fields who are 
experienced and qualified in multimedia design process. 

The research was designed in three phases as illustrated in figure 1. The first phase, was preparing the list of 
principles based on literature reviews and selection of experts. The second phase, a modified Delphi technique 
was used to facilitate experts’ opinion to reach consensus on the principles in designing and developing 
multimedia-based instructional media. The list of principles divided into three categories which are Basic 
principles, Authoring principles and Design principles. Basic principles comprise learning theories, 
instructional design theories and instructional design process. Authoring principles on the other hand, include 
authoring and software support tools. Design principles involve interactivity, screen design, audio and visual 
design and content design. The phase was the interview which was done twice with the three experts from the 
respective fields. Cognitive task analysis (CTA) uses a variety of interview and observation strategies to capture 
a description of the knowledge that experts use to perform complex tasks (Clark et. al, 2007). The first 
interview was done simultaneously with gaining experts’ consensus during the second round of the modified 
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Delphi technique. The second interview was carried out after analyzing the final consensus among the experts, 
hence the questions asked during the interview were based on the final consensus. 



Figure 1: Research Design 


4.0 RESEARCH FINDINGS 
4.1 Modified Delphi Technique 

For the cognitive principles as described in table 1, graphic designers obtain no consensus with high median 
score between 4 to 5 and interquartile range between 2.50 to 3.50. On the contrary, subject matter experts and 
instructional designers reach consensus with high median score 4 to 5 and interquartile range between 0.75 to 
1.50 except for elaboration theory. 


Table 1 : Consensus among subject matter experts, graphic designers and instructional designers for 

principles under cognitive theory. 


PRINCIPLES 

GD 


ID 


SME 


Med 

IQR 

Med 

IQR 

Med 

IQR 

prior knowledge 

4.5 

3.25 

4.5 

1.50 

5 

1.50 

hierarchical cognitive 

4.5 

2.50 

4 

0.75 

4 

0.75 

meaningful learning 

4.5 

3.25 

4.5 

1.00 

5 

0.75 

elaboration theory 

4.5 

3.25 

4.5 

3.25 

5 

2.25 

depth of processing 

4.5 

3.25 

5 

0.75 

4 

0.75 

conceptual model 

4.5 

3.25 

5 

0.75 

5 

0.75 

dual coding, symbol system 

4.5 

3.25 

5 

0.75 

4.5 

1.00 

invested mental effort 

4.5 

3.25 

4.5 

1.50 

4.5 

1.00 

cognitive-spatial maps 

4.5 

3.25 

4.5 

1.50 

5 

1.50 

situated and generative learning 

4 

3.00 

5 

0.75 

5 

0.75 

cognitive flexibility 

4 

3.00 

5 

0.75 

5 

1.50 
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For principles under behaviorism theory in table 2, graphic designers obtain no consensus with median score 
between 4 to 5 and interquartile range between 2.25 and 3.25. In contrast, subject matter experts and 
instructional designers obtain consensus with median score between 4 to 5 and interquartile range between 0.00 
to 1.50. 

Table 2 : Consensus among subject matter experts, graphic designers and instructional designers for principles 

under behaviorism theory 


PRINCIPLES 

GD 


ID 


SME 


Med 

IQR 

Med 

IQR 

Med 

IQR 

categorized material 

5 

3.00 

4 

0.00 

4 

0.75 

positive and negative examples 

5 

3.00 

5 

1.50 

4 

1.50 

write sequences 

4.5 

3.00 

5 

1.00 

5 

1.00 

sequence by difficulty 

4 

3.50 

4 

1.50 

4 

1.00 

sequence without learner control 

4 

3.25 

5 

2.00 

4.5 

2.00 

go through based on performance 

5 

3.00 

5 

1.00 

5 

1.00 

practice based on performance 

4 

3.25 

5 

1.50 

4 

1.50 

categorized skill with explanation 

5 

3.00 

5 

0.00 

5 

0.75 

performance standards are explicit 

5 

3.25 

4 

1.00 

4 

1.50 

practice build proficiency 

4.5 

3.25 

4 

0.75 

4 

0.00 

use of remedial loop 

4.5 

3.00 

5 

1.00 

5 

1.00 

reinforcement for motivation 

5 

3.00 

4 

0.75 

5 

0.75 


Table 3 : Consensus among multimedia designers, graphic designers and instructional designers for principles 

under constructivism theory 


PRINCIPLES 

GD 


ID 


SME 


Med 

IQR 

Med 

IQR 

Med 

IQR 

discovery learning 

5 

3.00 

5 

1.00 

5 

1.00 

scaffolding 

5 

2.75 

4 

0.75 

5 

0.75 

authentic task 

5 

2.75 

5 

0.75 

5 

0.75 

multiple intelligence 

5 

3.00 

4.5 

1.00 

5 

1.00 

stages of development 

4 

3.00 

4 

3.00 

5 

3.00 

multiple reality representation 

4 

2.75 

5 

2.25 

5 

2.25 

reflection 

5 

3.00 

5 

3.50 

5 

3.50 

collaborative learning 

5 

3.00 

4.5 

0.75 

5 

0.75 

learner centered design 

5 

3.00 

5 

0.75 

5 

1.00 

intrinsic motivation 

5 

3.00 

4.5 

0.75 

5 

1.00 

active learning 

5 

3.00 

5 

1.00 

5 

1.00 


As described in table 3, the three groups of experts obtain no consensus for the principles such as multiple 
reality representation and reflection with median score from 4 to 5 and interquartle range from 2.25 to 3.50. 
Subject matter experts and instructional designers reach consensus with high median score between 4 to 5 and 
interquartile range between 0.75 to 1.00. However, graphic designers obtain no consensus for all the principles 
with median score from 4 to 5 and interquartle range from 2.75 to 3.00. 

Table 4. : Consensus among multimedia designers, graphic designers and instructional designers for principles 

under andragogy theory 


PRINCIPLES 

GD 


ID 


SME 


Med 

IQR 

Med 

IQR 

Med 

IQR 

Provide learning outcomes 

5 

2.00 

5 

0.00 

4 

1.00 

Self-directed learning 

5 

3.00 

5 

0.75 

4 

1.00 

Experiential learning 

4.5 

2.00 

5 

0.75 

4 

1.00 

Cope effectively with real-life situation 

5 

2.00 

5 

0.75 

4 

1.00 

Applicable 

4.5 

3.00 

5 

0.00 

4 

1.00 

Internal pressures 

5 

2.00 

5 

0.75 

4 

1.00 


Table 4 described that, graphic designers obtain no consensus with median score from 4.5 to 5 and interquartile 
range between 2.00 to 3.00. Instructional designers and subject mater experts obtained high consensus with 
median score 4 to 5 and interquartile range from 0.00 to 1.00. 
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Table 5 : Consensus among subject matter experts, graphic designers and instructional designers for principles 

under instructional design theory 


PRINCIPLES 

GD 


SME 


ID 



Med 

IQR 

Med 

IQR 

Med 

IQR 

advance organizes & learning hierarchies 
feedback and emerging technologies 

4.5 

3.25 

5 

2.25 

5 

0.75 

generic prompts 

4.5 

3.25 

4.5 

3.25 

5 

0.75 

instructional control 

4.5 

3.25 

5 

3.25 

5 

0.75 

structural cueing 

4.5 

3.25 

4.5 

3.50 

5 

0.75 

navigational aids 

4.5 

3.25 

5 

2.25 

5 

0.75 

feedback 

4 

3.25 

4 

3.25 

5 

0.75 

mastery learning 

4.5 

3.25 

4.5 

3.25 

5 

0.00 

problem-based learning 

4 

2.00 

4 

2.00 

5 

0.00 

cooperative learning 

4 

2.25 

4 

2.25 

5 

0.75 

Needham model 

4 

3.00 

4 

3.00 

5 

0.75 

Laurillard conversational framework 

4 

3.00 

5 

3.00 

5 

0.75 

drill and practice 

4 

2.25 

4 

2.25 

5 

2.00 

tutorial 

4 

2.25 

4 

0.75 

5 

0.00 

simulation 

4 

2.25 

5 

0.75 

5 

0.00 

instructional game 

4 

3.00 

4 

0.75 

5 

0.00 

problem solving 

4 

3.00 

4 

0.75 

5 

0.00 

ARCS 

4 

3.00 

4 

0.75 

5 

0.00 

ABCD 

5 

3.00 

5 

1.00 

5 

0.75 


5 

3.00 

5 

1.00 

5 

0.75 


Table 5 explains that the experts obtain no consensus with regard to laurillard conversational framework with 
median score between 4 to 5 and interquartile range from 2.00 to 2.25. Subject matter experts obtain no 
consensus for majority of the principles with median score between 4 to 5 and interquartile range from 2.00 to 
3.50. Graphic designers obtain no consensus for all the principles with high median score between 4 to 5 and 
high interquartile range between 2.00 to 3.25. On the other hand, Instructional designers obtain high consensus 
with median score 5 and interquartile range between 0.00 to 0.75 with the rest of the principles. 

Table 6: Consensus of Subject Matter Expert, Graphic Designer and Instructional designer in software support 

tools 


PRINCIPLES 

GD 

ID 

SME 

Med 

IQR 

Med 

IQR 

Med 

IQR 

graphic tools 

5 

0.00 

5 

3.50 

4 

3.00 

planning and organization content- 

5 

0.75 

5 

3.50 

4 

3.25 

area tools 

5 

0.00 

5 

3.50 

4 

3.25 


Instructional designers and subject matter obtain no consensus for all the principles in software support tools as 
illustrated in table 6. The median score obtain is between 4 to 5 and interquartile range between 3.00 to 3.50. In 
contrast, graphic designers obtain high consensus for software support tools with median score 5 and 
interquartile range between 0.00 to 0.75. 

Table 7 : Consensus of subject matter expert, graphic designer and instructional designer in interactivity 

PRINCIPLES 


GD 


ID 

SME 



Med 

IQR 

Med 

IQR 

Med 

IQR 

provide opportunities 

4 

3.00 

4 

0.75 

5 

0.00 

chunk content 

4 

3.25 

5 

0.75 

4 

0.75 

question by content 

4 

2.00 

4 

0.75 

5 

0.75 

question prior knowledge 

4 

3.00 

5 

0.75 

5 

0.00 

apply what is learned 

5 

2.25 

4 

0.00 

4 

0.00 

rhetorical questions 

5 

3.25 

4 

0.75 

4 

0.75 

active exploration 

5 

2.00 

5 

0.75 

5 

0.00 
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As described in table 7, instructional design and subject matter expert similarly obtained consensus for 
principles under interactivity with high median score between 4 to and low interquartile range between 0.00 to 
0.75. However, graphic designers obtain no consensus for all the principles with median score between 4 to and 
interquartile range is between 2.00 to 3.25. 

Table 8 explains that graphic and instructional designers obtain no consensus for all the principles under 
analysis phase. The median score obtain between 4 to 5 and interquartile range between 2.00 to 3.25. In 
contrast, subject matter experts obtain high consensus for the principles under analysis phase. The median score 
is between 4 to 5 with low interquartile range between 0.00 to 0.75. Graphic and subject matter experts obtain 
no consensus for all the principles under the design phase. The median score obtain between 4 to 5 and 
interquartile range between 2.00 to 3.50. In contrast, instructional designers obtain high consensus for all the 
principles under the design phase. The median score is 4.5 with low interquartile range between 0.00 to 0.75. 
Instructional designers and subject matter experts obtain no consensus for all the principles under develop 
phase. The median score obtain between 4 to 5 and interquartile range between 2.25 to 3.00. However, graphic 
designers obtain high consensus for all the principles under develop phase. The median score is 4 with low 
interquartile range 0.75. Instructional designers and subject matter experts obtain no consensus for all the 
principles under produce phase. The median score was between 4 to 5 and interquartile range between 3.00 to 
3.50. Nevertheless, graphic designers obtained high consensus for all the principles under the produce phase of 
the instructional design process. The median score is 5 with low interquartile range between 0.00 to 0.75. All 
the experts obtain no consensus with regard to all the principles involve in the authoring phase. The median 
score was between 4 to 5 and interquartile range between 2.00 to 3.50. They also obtain no consensus with 
regard to all the principles involve in validation phase. The median score was between 4 to 5 and interquartile 
range between 2.25 to 3.25. All the three groups of experts obtain high consensus for enhancement phase with 
high median score between 4 to 5 and low interquartile range between 0.00 to 0.75. Similarly, they obtain high 
consensus for maintenance phase with high median score between 4 to 5 and low interquartile range between 
0.00 to 0.75. 
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Table 8 : Consensus among subject matter experts, graphic designers and instructional designers for principles 

instructional design process 


PRINCIPLES 

GD 


ID 


SME 


Med 

IQR 

Med 

IQR 

Med 

IQR 

Analysis Phase 







define problem 

4 

3.00 

4 

2.00 

4 

0.75 

describe facilities & schedule 

5 

3.25 

5 

2.00 

5 

0.75 

audience detail description 

5 

3.00 

5 

2.00 

5 

0.75 

needs analysis 

5 

3.00 

5 

2.00 

5 

0.75 

general objectives 

5 

2.25 

5 

2.00 

5 

0.00 

Design Phase 







evaluate team capability 

4 

3.50 

4.5 

0.75 

5 

2.00 

review objectives 

5 

3.00 

4.5 

0.00 

5 

2.00 

plan high level design process 

4 

3.25 

4.5 

0.75 

5 

2.00 

prepare design strategies 

5 

3.00 

4.5 

0.00 

5 

2.00 

Develop Phase 







develop storyboard 

4 

0.75 

5 

3.00 

4 

3.00 

develop scripts 

4 

0.75 

5 

3.00 

4 

3.00 

write for narrators 

4 

0.75 

5 

3.00 

4 

2.25 

computer screens and video 

4 

0.75 

5 

3.00 

4 

3.00 

graphics plan budget 

4 

0.75 

5 

3.00 

4 

3.00 

Produce Phase 







organize materials 

5 

0.00 

4 

3.25 

5 

3.00 

check content accuracy 

5 

0.75 

4 

3.50 

4.5 

3.25 

edit and evaluate media 

5 

0.75 

4 

3.50 

5 

3.50 

components 







Author Phase 







integrate media elements 

4 

2.00 

5 

3.00 

4 

3.50 

end users’ perspectives 

5 

2.00 

5 

3.25 

4 

3.50 

refine the presentations 

5 

2.00 

5 

3.25 

4 

3.50 

Validate Phase 







measurement instrument 

5 

3.00 

4 

2.25 

4 

2.25 

interview audience and record 

5 

3.25 

5 

3.25 

5 

3.25 

analyze validation findings 

5 

3.25 

5 

3.00 

5 

3.00 

Enhancement Phase 







beautify 

4 

0.00 

5 

0.75 

5 

0.75 

Maintenance Phase 







maintain 

4 

0.00 

4 

0.75 

5 

0.75 

Table 9 : Consensus of subject matter expert, graphic designer and instructional designer 

in screen design 

PRINCIPLES 


GD 


ID 

SME 


Med 

IQR 

Med 

IQR 

Med 

IQR 

prepare for learning 

4 

0.75 

5 

0.00 

4.5 

0.00 

develop and maintain interest 

5 

0.00 

5 

0.75 

4 

0.75 

deep processing 

5 

2.00 

5 

1.00 

4 

1.00 

learner engagement 

5 

1.00 

5 

0.75 

4 

0.75 

organize information 

4 

1.75 

5 

0.75 

5 

0.75 

facillitate lesson navigation 

4 

1.75 

5 

1.00 

5 

1.00 

learning environment 

4.5 

2.00 

4 

0.75 

5 

0.75 

understood 

5 

1.00 

4 

0.00 

4.5 

0.00 

familiar 

5 

1.00 

4 

0.00 

4.5 

0.00 

minimal cognitive resources 

5 

2.00 

4.5 

0.75 

5 

0.75 


Instructional designers and subject matter experts gain consensus for all the principles in screen design with 
high median score between 4 to 5 and low interquartile range between 0.00 to 1.00 as explained in table 9. 
However, graphic designers obtain no consensus for principles such as deep processing, learning environment 
and minimal cognitive resources with high median score between 4.5 to 5 and interquartile range 2.00. 
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Table 10 : Consensus among subject matter expert, graphic designer and instructional designer in authoring 

tools 


PRINCIPLES 


GD 

ID 

SME 

Med 

IQR 

Med 

IQR 

Med 

IQR 

HyperCard and Linkway 

5 

2.75 

4 

2.25 

4 

2.25 

Powerpoint 

5 

3.00 

4 

2.25 

4 

2.25 

video as real-time communication 

4 

2.00 

5 

3.50 

4 

2.00 

video to depict problem 

4 

2.00 

4 

2.25 

4 

2.25 

weblog 

4 

2.00 

4.5 

3.00 

5 

3.50 

Wikis 

5 

3.00 

4 

2.50 

4 

2.50 

podcast 

4 

2.25 

5 

3.00 

5 

3.25 

e-portfolios 

4 

3.00 

5 

2.00 

5 

2.00 

video sharing communities 

4 

2.25 

5 

2.25 

4 

2.75 

social networking sites 

4 

2.25 

5 

3.00 

5 

3.00 

Quicktime Virtual Reality 

4 

3.50 

5 

3.25 

5 

2.00 

Geographic Information Systems 

5 

2.25 

4 

2.00 

4 

2.00 

3-D models 

5 

3.00 

4 

2.00 

4 

2.00 

virtual reality environment 

5 

2.50 

5 

3.25 

5 

3.00 

full immersion system 

4 

3.00 

4 

3.00 

5 

2.25 

Facebook 

4 

3.00 

4 

3.50 

5 

3.00 


Table 10 shows that the experts similarly obtain no consensus for principles under authoring tools. They obtain 
high median score between 4 to 5 and interquartile range between 2.00 to 3.50. 

Table 11 :Consensus among subject matter expert, graphic designer and instructional designer for principles 

under content design 


PRINCIPLES 


GD 

ID 

SME 

Med 

IQR 

Med 

IQR 

Med 

IQR 

multimedia principle 

5 

3.00 

5 

3.00 

4 

3.00 

split attention principle 

5 

2.75 

5 

0.00 

4 

0.75 

modality principle 

5 

3.00 

5 

0.75 

4 

0.75 

redundancy principle 

5 

2.00 

5 

0.75 

4 

0.75 

segmenting principle 

5 

2.75 

5 

0.75 

4 

0.00 

pretraining principle 

5 

3.00 

5 

0.00 

5 

0.00 

coherence principle 

5 

3.00 

5 

0.00 

5 

0.00 

signaling principle 

4 

0.75 

5 

0.00 

5 

0.00 

spatial contiguity principle 

5 

2.00 

5 

0.75 

5 

0.75 

temporal contiguity principle 

5 

2.75 

5 

0.00 

5 

0.00 

voice principle 

5 

3.75 

5 

0.00 

5 

0.00 

personalization principle 

5 

2.75 

5 

0.00 

5 

0.00 

cognitive aging principle 

5 

3.00 

5 

0.00 

5 

0.00 

prior knowledge principle 

5 

3.00 

5 

0.00 

5 

0.00 

site map principle 

5 

0.75 

5 

1.75 

5 

1.75 

navigation principle 

5 

0.75 

5 

1.00 

4 

1.00 

guided-discovery principle 

5 

2.00 

5 

0.75 

4 

0.75 

worked-out example principle 

5 

2.00 

5 

0.75 

5 

0.75 

collaboration principle 

5 

2.00 

5 

0.75 

4 

0.75 

self-explanation principle 

5 

2.00 

5 

0.75 

4 

0.75 

animation principle 

5 

1.00 

5 

3.25 

4 

3.50 

interactivity principle 

5 

1.00 

5 

0.75 

4 

0.75 

Self-critique principles 

5 

0.75 

5 

0.00 

5 

0.00 

Conceptual principles 

5 

0.75 

5 

0.00 

4 

0.00 

Marketability principles 

5 

0.75 

5 

0.00 

4 

0.00 

Consistency principles 

5 

0.75 

5 

0.00 

5 

0.00 


The experts obtain consensus for the principles such as self-critique principles, conceptual principles, 
marketability principles, consistency principles, site map principle, navigation principle, interactivity principle 
and signaling principle with high median score between 4 to 5 and interquartile range between 0.00 to 1.00. 
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They similarly obtain no consensus for principles such as multimedia with high median score between 4 to 5 
and interquartile range 3.00. Subject matter experts and instructional designers obtain high consensus for 
majority of the principles with median score between 4 to 5 and interquartile range between 0.00 to 0.75. 
Moreover, they also similarly obtain no consensus for animation principle with median score between 4 to 5 and 
interquartile range between 3.25 and 3.50. Graphic designers obtain no consensus for majority of the principles 
with median score between 4 to 5 and interquartile range between 2.00 to 3.00. 


Table 12: Consensus among subject matter expert, graphic designer and instructional designer for principles 

under audio visual 


PRINCIPLES 


GD 

ID 

SME 

Med 

IQR 

Med 

IQR 

Med 

IQR 

color is used sparingly 

4 

0.75 

5 

3.25 

4.5 

2.75 

color for cue 

5 

0.00 

4 

2.00 

4 

3.00 

color for highlight 

4 

0.00 

5 

2.75 

5 

2.00 

color for important points 

5 

0.00 

5 

3.00 

5 

2.00 

consistent color scheme 

4 

0.75 

4 

2.00 

5 

2.00 

graphics and difficult topics 

5 

0.75 

4 

2.00 

5 

3.00 

graphics illustrates concept 

4 

0.75 

5 

2.00 

5 

2.25 

animation for key animation 

4 

1.75 

5 

3.00 

4 

3.00 

animation and students' interest 

5 

1.00 

5 

2.25 

4 

2.25 

animation facilitate recall 

5 

1.75 

5 

3.00 

4 

3.00 

audio and visual presentation 

5 

1.00 

5 

3.50 

4 

2.50 

video for advance organizer 

5 

2.00 

5 

2.25 

5 

3.00 

video for lesson summarization 

5 

2.75 

5 

3.00 

5 

2.00 

principles of art 

5 

0.75 

5 

2.50 

5 

2.25 

elements of design 

4 

0.75 

5 

3.00 

4 

3.00 

principles of organization 

5 

0.75 

4 

3.00 

4 

3.00 

compositional techniques 

5 

0.00 

4 

2.25 

5 

2.50 


Instructional designers and subject matter experts obtain no consensus for all the principles under audio visual 
with high median score between 4 to 5 and interquartile range between 2.00 to 3.00. Graphic designers obtain 
consensus for majority of the principles with median score between 4 to 5 and interquartile range between 0.00 
to 1.75. They obtain no consensus pertaining to video for advance organizer and video for lesson 
summarization with high median score 5 and interquartile range between 2.00 to 2.75. 

4.2 Interview 

Findings from the interview of an expert in the respective field are categorized into shared task and expertise- 
based task. 

4.2.1 Shared Task 

The findings highlight the importance of sharing ideas among members of the multimedia design team. Expert 
01 emphasizes that discussion is important where graphic designers share ideas with other members of the team 
through drawing or writing in explaining the storyboard. The visualization or discussion clarifies uncertainties 
among team members. Due to that, text as well as illustrations are applied in order to ensure the message 
delivered is clear and precise. Expert 03 further adds that graphic designers should not obstruct or constrain 
themselves from contributing ideas. 

graphic designer cannot be stingy in giving ideas ...if committed graphic designer ...they will 
think this way... they know that it is difficult for teachers to teach certain concept so we use mm 
to explain to students effectively ...so do not be stingy throw ideas for other partners to know... 
in team work manager is important. 

Expert 01 stresses that if graphic designers are unable to explain in words they are welcome to draw their ideas 
as long as the ideas are shared among members. Thus, sharing ideas in order to ensure the quality of the product 
is no doubt essential for every member of the team. Expert 01 elaborates: 

We don 7 call it meeting....but we call it visualization..meaning we have to visualize first ... like 
brainstorming or mindmapping ... during the discussion there’ll be a white board ... graphic 
designers start sketch what the discussion about..they use thumbnail... either he’ll draw or 
write.... thumbnail is small boxes ... and this is the basic for storyboard.. 
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Instructional designers and subject matter experts share knowledge in the application of screen design 
principles. They similarly involve in application of majority of the principles under content design. Their 
specialization in developing and designing content requires them to apply principles under cognitivism, 
behaviorism, constructivism, andragogy, instructional design theory and interactivity in designing multimedia. 
This is elaborated by expert 01 that: 

subject matter expert developing content..instructional designer focusing on design aspect.. 

The two parts a and c, all three of them as, share the same viewpoint for the principles under authoring tools 
that the tasks requires technical skills which does not involve job specifications of subject matter experts, 
instructional designers and graphic designers. 

4.2.2 Expertise-based Task 

From the findings, some of the applications are not shared but applied according to area of specialization. 

Instructional designers’ and subject matter experts’ task in instructional design process are segregated. Subject 
matter experts focus on the analysis phase where they define problem, describe facilities and schedule, analyze 
learners’ detailed description, needs analysis and general objectives. On the other hand, instructional designer 
concentrate on designing aspects such as planning high level design process, prepare design strategies, evaluate 
team capability and review objectives. This is further advocated by expert 01 who explains that: 

the flow of work starts from subject matter expert developing content..instructional designer 
focusing on design aspect. . 

Unlike instructional designers and subject matter experts, graphic designers do not involve in the application of 
principles under cognitivism, behaviorism, constructivism, andragogy, instructional design theory and 
interactivity in designing multimedia. Graphic designers involve totally in application of the principles under 
audio visual and software support tools which is not applicable among instructional designers and subject matter 
experts. As for instructional design principles, graphic designers involve in develop phase produce phase. 

Expert 01 elaborates that graphic designers perform task based on their observation of characters, images and 
events around them. Their work is mostly based on experience rather than reading materials. Expert 01 explains: 
the role of graphic designers is not just reading..he obtain information by digesting his 

surrounding . meaning he needs to observe characters, images..reading or reference as such 

are not applicable and very limited for graphic designers .. 

Segregation of tasks will be easier and all members will perform the best according to their potential. This will 
help to ensure product quality. Expert 02 explains: 

so when we interact with people we know how they perform so we assign task according to 
their nature for example a person who is detail in doing his work so we assign him the difficult 
part for him to complete ...whereas the person who has the skill but cannot achieve the level 
that we want ...so assign him with simpler task ...when dividing task as a leader we have to 
follow according to potential of the person ... 

5.0 DISCUSSION 
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Figure 2 : Proposed Shared Mental Model of Multimedia Design Experts in Designing Instructional Media 

Model of shared mental among instructional designers, graphic designers and subject matter experts is suggested 
based on the research finding. The model highlights four parts the shared knowledge between graphic designers 
and instructional designers (IG), instructional designer and subject matter expert (IS), graphic designer and 
subject matter expert (GS) and shared knowledge among the experts (SK) as shown in figure 2. 

Instructional designers and subject matter experts share knowledge and ideas in most of the application of 
principles in designing multimedia. Jamalludin et al. (2001) categorized the team according to three major 
groups; management, design and technical. The design group includes subject matter expert, instructional 
designer and script writer. This is also supported by Alessi and Trollip (2001) who divide the team into a group 
that is to create a design of the program that leads to the most effective learning by the target audience and 
another group is to produce or oversee the production of a set of documents that communicate effectively with 
the rest of the team. The team inclusive of instructional designer, content experts or subject matter expert, 
clients, project manager, trainers or teachers and learners. 

Based on the findings, the shared knowledge involves the principles under content design such as self-critique 
principles, conceptual principles, marketability principles, consistency principles, site map principle, navigation 
principle, interactivity principle and signaling principle. Apart from that all three of them do not involve in 
author and validate phase. Lee and Owens (2000) explain that validation phase involves Quality Review 
(Evaluator). Even though enhancement and maintenance phases are newly added phases in the instructional 
design process, all the three groups of experts agree that those two phases are important. The additional two 
phases are similar to the Three-Phase Development (3PD) Model. Sims and Jones (2003) elaborate that the 
phases elicit learning content through process of enhancing and maintaining materials, rather than the more 
traditional systems approach of analyze, design, develop, implement, evaluate. Subject matter experts, 
instructional designers and graphic designers similarly do not apply principles under authoring tools, video for 
advance organizer and video for lesson summarization. The result is due to the principles under authoring tools 
requires technical skills which does not involve job specifications of subject matter experts and graphic 
designers. 

A systematic flow of performing tasks from one specialization to another in instructional design process is 
advocated by experts from the interview. They start with subject matter experts developing content, then pass 
their work to instructional designers for designing the instruction. Graphic designers illustrate concept. 
Jamalludin et al. (2001) categorized the team according to three major groups; management, design and 
technical. The collaboration of the three groups determine the flow of the project and thus ensures the quality of 
the multimedia product. 

The team members have a shared mental model of their own roles and functions. Hence, they have the same 
understanding for the dynamics of key processes. These processes can include the roles and functions of each 
team member in accomplishing the task, the nature of the task, the use of equipment and so forth (Schraagen et 
al., 2000). The involvement of each expert varies according to their specialization of tasks. Norazlin et al. 
(2007) agree to the notion of grouping the team into three different groups according to their area specialization. 
Bergman and Moore (1991) add that some individuals may have multiple skills, thus the team does not 
necessarily comprises of every specialization. Careful selection of members with the right skills is essential as it 
is not realistic to expect any one member to possess all the sophisticated skills required. 

6.0 CONCLUSION 

Previous research (Keppell, 2000) addresses the gap in the field of instructional design and outlines a number of 
key principles to consider in interacting with subject matter experts. Without effective principles for interacting 
with the subject matter expert, valuable time will be lost understanding and organizing the content. Norain and 
Siti Salwah (2012) developed guidelines for developing e-learning storyboard for effective ID-SME interaction. 
This research extends the scope of specialization, analyzing cognitive task of graphic designers in designing 
multimedia. The shared knowledge among the experts emerges as solid understanding of the factors that 
influence team decision making and performance in order to identify interventions that can affect the decision 
making process and improve performance (Hall & Regian, 1996; Klinger et al., 1993; Salas, Bowers, & 
Cannon-Bowers, 1995). 


Copyright © The Turkish Online Journal of Educational Technology 


167 





TOJET: The Turkish Online Journal of Educational Technology - July 2013, volume 12 issue 3 



REFERENCES 

Alessi, S.M. & Trollip, S.R (2001). Multimedia for Learning: Methods and Development Boston: Allyn and 
Bacon. 

Bergman, R. E., Moore, T. V. (1991). Managing Interactive Video/Multimedia Projects. Englewood Cliffs, 
New Jersey: Educational Technology Publications. 

Cooke, N. J., Shope, S. M., & Kiekel, P.A. (2001). Shared-Knowledge and TeamPerformance: A Cognitive 
Engineering Approach to Measurement. Technical Report for AFOSR Grant No. F49620-98-1-0287. 

Feltovich, P.J., Prietula, M.J., Ericsson, K.A., (2006). Studies of expertise from psychological perspectives. In: 
Ericsson, K.A., Chamess, N., Feltovich, P.J., 

Hoffman, R.R. (Eds.), The Cambridge handbook of expertise and expert performance. Cambridge University 
Press, Cambridge, U.K., pp. 41-68. 

Hall, E. and Regian, W. (1996). Cognitive engineering for team tasks. AFOSR Laboratory Research Initiation 
Request. Retrieved January 2010 from 

http ://www. dodccrp. or g/ events/6th_ICCRTS/Tracks/Papers/T rack2/041 _tr2 

Hoffman, R. R. & Militello Laura G. (2008). Perpectives on cognitive task analysis: Historical origins and 
modern communities of practice. U.S.A.: Taylor & Francis Group. 

Jamalludin Harun, Baharuddin Aris & Zaidatun Tasir. (2003). Pembangunan Perisian Multimedia: Satu 
Pendekatan. Selangor: Venton. 

Keppell, M. (2000). Principles at the Heart of an Instructional Designer: Subject Matter Expert Interaction. 
Ascilite.org.au. Retrieved January 2000 from 
http://www.ascilite.org.au/conference/papers/mike_keppell.pdf. 

Klinger, D. W., Andriole S. J., Militello L. G., Adelman, L., Klein, G., and Gomes, M. E. (1993). Designing for 
performance: A cognitive systems engineering approach to modifying an AW ACS human computer 
interface. Armstrong Laboratory, AL/CFTR-1993-0093, Wright-Patterson AFB, OH. 

Lee, W. W. & Owens, D. L (2000). Multimedia-Based Instructional Design: Computer-Based Training, Web- 
Based Training & Distance Broadcast Training. USA.: Wiley. 

Martino, J. P. (1972). Technological Forecasting for decision making. New York: American Elsevier 
Publishing. 

Nor'ain, M. Y., & Siti Salwah, S. (2012). Investigating Cognitive Task Difficulties and Expert Skills in e- 

Leaming Storyboards Using a Cognitive Task Analysis Technique. Computers & Education, 58(1), 
652-665 

Norazlin Mohammed, Ibrahim Ahmad, Mohd. Hafiz Zakaria, Muhammad Haziq Lim Abdullah, Farah Nadia 
Azman, Shahrul Badariah Mat Sah, Rusnida Romli, Noraksiken Bakar, Faaizah Shahbodin. (2007). 
Sistem Multimedia. Selangor: Venton Publishing. 

Norman G. R. and Brooks L. R. (May, 1997). The Non-Analytical Basis of Clinical Reasoning Advances in 
Health Sciences Education Springer Netherlands Vol. 2, No. 2. 

Salas, E., Bowers, C.A., and Cannon-Bowers, J.A. (1995). Military team research: lOyears of progress. Military 
Psychology , 7, 5575. 

Schmidt H.G. & Boshuizen H. P. A. (2002). On the Constraints of Encapsulated Knowledge: Clinical Case 
Representations by Medical Experts and Subexperts, 2002 Taylor & Francis, Ltd ..Cognition and 
Instruction, 20(1), 27-45, Earlbaum Association Inc. 

Schraagen, M., Chipman, S. F. & Shalin, V. L. (2000). Cognitive Task Analysis. Mahwah: NJ.: Lawrence 
Earlbaum Associates Inc. 


Copyright © The Turkish Online Journal of Educational Technology 


168 





